1. Field of the Invention
The present invention relates to a three-dimensional character image processing device and method for generating a realistic three-dimensional model by using an image, and to a recording medium for storing an image processing program.
2. Description of the Related Art
Conventionally, various techniques for generating a realistic three-dimensional (hereinafter referred to as xe2x80x9c3-Dxe2x80x9d) character image using a plurality of images have been proposed.
For example, in a 3-D model generation device disclosed in Japanese Laid-Open Patent Publication No. 2001-109907, a 3-D model is transformed so as to conform to an input image by designating positions of specific points in a face image included in the input image. A corresponding relationship between the input image and the 3-D model is calculated so as to generate a textured image such that the generated textured image includes a face image as viewed from the front and pixel values in the textured image are uniform both inside and outside of the outline of the face, thereby generating a natural 3-D character image.
Further, a 3-D model generation device disclosed in Japanese Patent Application No. 11-283387 (Japanese Laid-Open Patent Publication No. 2001-109913) is provided with a means for selecting models corresponding to various parts of a 3-D character image to be generated, so that a posture and a hairstyle of the character in the 3-D character image can be freely selected according to a user""s preference.
In the above-described conventional techniques, 3-D models corresponding to various parts, such as a face, hair, a body, etc., are provided so as to generate a realistic 3-D character image using a plurality of images. When generating a face image, calculations for converting the 3-D models into images are simultaneously performed so that positions of the various parts are not inconsistent in a depth direction.
This method requires a great number of calculations in the steps of rotating the models, pasting a textured image on each surface of the models, and realizing consistency of the model surfaces in the depth direction. Therefore, a storage device having a great capacity is required for the calculations. Therefore, in order to generate a realistic 3-D character image using this method, it is necessary to use a computing unit exclusively structured for image processing or a device which includes an image processing program specialized for use in a specific device and incorporated into a personal computer having a general-purpose computing function with high processing ability.
That is, the conventional method cannot be performed by a computing device or a mobile computer, which do not have a storage device with a great capacity, and therefore such a computing device or mobile computer cannot generate a realistic 3-D character image. Further, in the computing device or mobile computer with a relatively low processing ability, which do not have a computing unit for exclusive use in image processing, even if a problem related to the capacity of the storage device is solved, a great deal of time is spent for calculations, making a user of the device feel uncomfortable.
An image processing device for generating a 3-D character image of a target object included in an input image of the present invention includes: an image input section for inputting a face image; a storage section for prestoring plurality types of 3-D models of hair and faces, a plurality of body images, and information on a direction, position, and size of a face conforming to the body images; a user input section having a position designation means for designating an arbitrary position in the image input via the image input section and a body selection means for selecting the body image stored in the storage section; an image processing section, including a textured image generation means for generating, based on information representing a position designated by the position designation section, a textured image as viewed from a prescribed position and direction having a prescribed size so as to paste the input image onto the 3-D model, a textured image storage means for storing the textured image generated by the textured image generation section, a face image generation means for generating a 3-D still head image by rotating a 3-D face model according to the information on a direction, position and size of a face conforming to the body image, painting the hair, and pasting the textured image onto the 3-D face model, and a combining means for combining a body image selected by the body selection means and an image generated by the face image generation section so as to generate a 3-D character image; and a display section for displaying the 3-D character image generated by the image processing section.
An image processing method for generating a 3-D character image of a target object included in an input image of the present invention includes the steps of: a) inputting a face image via an image input section; b) storing in a storage section plurality types of 3-D models of hair and faces, a plurality of body images, and information on a direction, position, and size of a face conforming to the body images; c) designating an arbitrary position in the image input via the image input section; d) selecting the body image stored in the storage section; e) generating, based on information representing a position designated by step c), a textured image as viewed from a prescribed position and direction having a prescribed size so as to paste the input image onto the 3-D face model; f) storing the textured image generated by step d); g) generating a 3-D still head image by rotating 3-D face and hair models according to the information on a direction, position and size of a face conforming to the body image selected by step d), painting the hair, and pasting the textured image onto the 3-D face model; h) combining a body image selected by step d) and an image generated by step g) so as to generate a 3-D character image; and i) displaying the 3-D character image generated by step h).
A computer-readable recording medium of the present invention stores an image processing program for executing an image processing method for generating a 3-D character image of a target object included in an input image, and the method includes the steps of: a) inputting a face image via an image input section; b) storing in a storage section plurality types of 3-D models of hair and faces, a plurality of body images, and information on a direction, position, and size of a face conforming to the body images; c) designating an arbitrary position in the image input via the image input section; d) selecting the body image stored in the storage section; e) generating, based on information representing a position designated by step c), a textured image as viewed from a prescribed position and direction having a prescribed size so as to paste the input image onto the 3-D face model; f) storing the textured image generated by step d); g) generating a 3-D still head image by rotating a 3-D face model according to the information on a direction, position and size of a face conforming to the body image selected by step d), painting the hair, and pasting the textured image onto the 3-D face model; h) combining a body image selected by step d) and an image generated by step g) so as to generate a 3-D character image; and i) displaying the 3-D character image generated by step h).
With the above-described image processing device, method and recording medium of the present invention, a 3-D character image is generated by combining a face image, having an arbitrary direction, position and size, with a body image selected by the body selection means. The face image is generated by designating specific points in a face image input via the image input section using the position designation means and pasting a hair image and the input face image onto 3-D hair and face models. According to the present invention, initially the face image is input via the image input section and the specific points in the input face image are designated using the position designation means, and then a face image, which has a direction, position and size designated by calculations based on the input face image, is generated. The generated face image and a prestored body image are combined so as to generate a desired 3-D character image. In this manner, although the present invention uses calculations for generation of the face image, calculations related to each of hair and body images are reduced so as to be minimal, and therefore it is possible to generate a highly-realistic 3-D character image even using a computing device or mobile computer with relatively low processing ability.
Further, in the case where a plurality of time-series body images and information on a direction, position and size of a face conforming to the body images are provided, it is possible to generate a highly-realistic 3-D animated character image using time-series information on the motion of body and face images.
In one embodiment of the present invention, the storage section stores a plurality of hair and face models, and the user input section includes a model selection section for selecting the plurality of hair and face models stored in the storage section.
In one embodiment of the present invention, step b) further includes the steps of j) storing a plurality of hair and face models and k) selecting the plurality of hair and face models stored by step b).
Further, with the above-described image processing device, method and recording medium of the present invention, the storage section stores a plurality of hair and face models and the user input section includes the model selection means, and therefore a model can be selected so as to match the hairstyle and the shape of the face (e.g. a round, square or sharp face) of a person who is a subject of an input image, thereby generating a highly-realistic still or animated image which looks the same as that person.
In one embodiment of the present invention, the storage section prestores a plurality of body images, which vary with the lapse of time, and information on a direction, position and size of a face conforming to the body images; and the combining means generates a 3-D animated image by generating a plurality of still images based on the 3-D head image generated by the face image generation means, the plurality of body images, which vary with the lapse of time, and the information on a direction, position and size of a face conforming to the body images.
In one embodiment of the present invention, the storage section stores vector still image data and information on a direction, position and size of a face for generating a still body image and the combining means generates the still body image based on the vector still image data stored in the storage section and combines the 3-D still head image generated by the face image generation section and the still body image based on the information on a direction, position and size of a face for generating the still body image.
In one embodiment of the present invention, the storage section stores vector or vector graphics animated body image data and information on a direction, position and size of a face for generating an animated body image, the face image generation section generates a plurality of face images, and the combining means generates the animated body image based on the vector animated image data stored in the storage section and combines the plurality of face images generated by the face image generation section and the animated body image based on time-series information on a direction, position and size of a face stored in the storage section.
In one embodiment of the present invention, step b) includes prestoring a plurality of body images, which vary with the lapse of time, and information on a direction, position and size of a face conforming to the body images; and step h) includes generating a 3-D animated image by generating a plurality of still images based on the 3-D still head image generated by step g), the plurality of body images, which vary with the lapse of time, and the information on a direction, position and size of a face conforming to the body images.
In one embodiment of the present invention, step b) includes storing vector still image data and information on a direction, position and size of a face for generating a still body image; and step h) includes generating the still body image based on the vector still image data stored by step b) and combining the 3-D still head image generated by step g) and the still body image based on the information on a direction, position and size of a face for generating the still body image.
In one embodiment of the present invention, step b) includes storing vector or vector graphics animated body image data and information on a direction, position and size of a face for generating an animated body image; step g) includes generating a plurality of face images; and step h) includes generating the animated body image based on the vector animated image data stored by step b) and combining the plurality of face images generated by step g) and the animated body image based on time-series information on a direction, position and size of a face stored by step b).
Further, in the image processing device, method and recording medium of the present invention, by storing vector still image data or vector or vector graphics animated data in the storage section, an animated or still body image can be generated from the vector still image data or vector graphics animated data, and therefore the data amount can be significantly reduced. Even when the size of the still or animated image is changed, few quantization errors are caused by enlargement or reduction of the still or animated image, and therefore a high quality still or animated image can be generated.
In one embodiment of the present invention, the image processing device further includes a function of previewing a head image generated by the face image generation means and displayed on the display section for confirmation.
In One embodiment of the present invention, the image processing method further includes a function of previewing a head image generated by step g) and displayed on a display section for confirmation.
Further, in the image processing device, method and recording medium of the present invention, by providing a function of previewing a head image generated by the face image generation means and displayed on the display section for confirmation, even in the case where it takes time to generate a plurality of images, a process result can be confirmed by a user after the process is performed on a single image such that the plurality of images requested by the user are processed after the user""s satisfaction at the process result. Therefore, there is no case where the user is unsatisfied with the process results after having waited a long time required for the processing of the plurality of images and recalculations for generation of those images are required. This allows a reduction in the overall time required for generating an animated image.
In one embodiment of the present invention, even when a hair or face model is changed by the model selection means, by prestoring a textured image in the textured image storage section, the face image generation section can reproduce a head image based on the textured image stored in the textured storage section by rotating the changed hair or face model according to information on a direction, position and size of a face conforming to a body image.
In one embodiment of the present invention, even when a hair or face model is changed by step k), by prestoring a textured image in step f), step g) can reproduce a head image based on the textured image stored in step f) by rotating the changed hair or face model according to information on a direction, position and size of a face conforming to a body image.
Further, in the image processing device, method and recording medium of the present invention, by prestoring a textured image in the textured image storage section, even when a hair or face model is changed by the model selection means, it is not necessary to designate specific positions in the input image again, and therefore the amount of operations performed by the user can be reduced and calculations for generation of the textured image can be omitted, thereby reducing the overall time required for generating the images.
In one embodiment of the present invention, even when a body image is changed by the body selection means, by prestoring a textured image in the textured image storage section, the face image generation section can reproduce a head image based on the textured image stored in the textured storage section by rotating head and face models according to information on a direction, position and size of a face conforming to the changed body image.
In one embodiment of the present invention, even when a body image is changed by step d), by prestoring a textured image in step f), step g) can reproduce a head image based on the textured image stored in step f) by rotating head and face models according to information on a direction, position and size of a face conforming to the changed body image.
Further, in the image processing device and method and recording medium of the present invention, by prestoring the textured image in the textured image storage section, even when a body image is changed by the body selection means, it is not necessary to designate specific positions in the input image again, and therefore the amount of operations performed by the user can be reduced and calculations for generation of the textured image can be omitted, reducing overall time required for generating the images.
In one embodiment of the present invention, the face image generation section has an antialiasing image reduction f unction for generating a high quality head image by generating an image larger than a target image and reducing a size of the generated image.
In one embodiment of the present invention, step g) further includes an antialiasing image reduction function step for generating a high quality head image by generating an image larger than a target image and reducing a size of the generated image.
Further, in the image processing device and method and recording medium of the present invention, by providing to the face image generation section the antialiasing image reduction function for generating a high quality head image by generating an image larger than a target image and reducing a size of the generated image, it is possible to generate a high quality head image, and where the generated head image is compressed so as to save the head image as, for example, a JPEG image using DCT (Discrete Cosine Transform), a high frequency component in head image data can be suppressed, thereby reducing noise in the compressed data and the size of the compressed data.
Thus, the invention described herein makes possible the advantages of providing: an image processing device and method in which a realistic and complicated 3-D character image can be generated through easy operations using a computing device or mobile computer with a relatively low processing ability, which does not have a computing unit for exclusive use in image processing nor a specialized image processing program; and a recording medium for storing an image processing program.
These and other advantages of the present invention will become apparent to those skilled in the art upon reading and understanding the following detailed description with reference to the accompanying figures.